Monitoring Message-Passing Parallel Applications in the Grid with GRM and Mercury Monitor

Application monitoring in the grid for parallel applications is hardly supported in recent grid infrastructures. There is a need to visualize the behavior of the program during its execution and to analyze its performance. In this paper the GRM application monitoring tool, the Mercury resource and job monitoring infrastructure and the combination of the two as a grid monitoring tool-set for message-passing parallel applications is described. By using them, one can see and analyze on-line the behavior and performance of the application. 1 Application monitoring in the grid There are several parallel applications that are used on a single cluster or a supercomputer. As users get access to an actual grid they would like to execute their parallel applications on the grid instead of the local computing resource for several reasons. Local resources are always limited in size and availability. They might not be capable of executing an application with a given size of input data so the user should look for a larger resource. Another limitation is that the local resource is likely to be occupied by other users for days or weeks and we may need to run our application today. In current grid implementations, we are already allowed to submit our parallel application to the grid and let it execute on a remote grid resource. However, those current grid systems are not able to give detailed information about our application during its execution except its status, like standing in a queue, running, etc. If we are interested in getting information about our application (how it is working, what its performance is) we realize that there is no way to collect such information. Performance analysis tools are not available yet for current grid systems. Our target of research has been to provide a monitoring tool that is able to collect trace information about an – instrumented – parallel application executed on a remote grid resource. This goal is different from providing a monitoring tool for metacomputing applications running on several resources simultaneously, i.e. being distributed applications. The latter goal requires the tool to be able to monitor processes on more than one resource at the same time that brings other requirements for the tool. Combining our GRM and Mercury Monitor tools we have achieved our goal and created an infrastructure that enables the user to collect performance information about The work described in this paper has been supported by the following grants: EU-DataGrid IST-2000-25182 and EU-GridLab IST-2001-32133 projects, the Hungarian Supergrid OMFB00728/2002 project, the IHM 4671/1/2003 project and the grant OTKA T042459.